1. Field of the Invention
The present invention relates to image processing, and more particularly, to a method and apparatus for image-based photorealistic 3D face modeling.
2. Description of the Related Art
Automated photorealistic 3D face modeling is considered as a very important task in the computer graphics field due to its wide applicability in various fields such as virtual space creation, computer gaming, video conferencing and animation.
Although a laser scanner capturing an accurate 3D image from a complicated object and a distance measuring apparatus using structured light are currently available, the cost is still high and it is not easy to use them. Computer-aided inter active face modeling technologies require professional, hard labor. So, intensive efforts have been devoted to allow common PC users to more easily reconstruct more realistic human face models. In other words, it have been required to develop an algorithm that can be easily controlled by a common PC user, to enable automated construction of desirable faces from imperfect input data, which are obtained using economical imaging devices such as digital cameras.
Systems for automated face model reconstruction can be roughly divided into several categories according to the used source of 3D face shape data. Some researchers developed techniques to acquire face data from laser scanners or structured light installations. Problems arising with these techniques lie in that the apparatuses are expensive and are difficult to use.
Many efforts have been devoted to create face models from frontal and profile views. However, the strictly orthogonal relation between the two images is difficult to achieve when using a common handheld camera without a special auxiliary device, so the shape information obtained directly from different views has some confliction. This problem has not been solved yet.
Some systems rely on user-specified feature points in several images, which require labour-intensive manual procedures and much time. Among other approaches, methods utilizing optical flow and stereo methods seem to make the farthest step towards full automation of the reconstruction process, but the resulting models include considerable noise and unnaturally deformed face surface.
To avoid undesirable artifacts of the face model and increase robustness, some researchers utilize a morphable face model, which allows use of a limited number of feature points. The morphable face model is matched with reconstructed 3D points or source images and requires model-based bundle adjustment for shape recovery. These approaches greatly rely on how accurate and extensive the set of feature points is.
In another method for obtaining an accurate model using stereo reconstruction, shape recovery is limited only to a deformable model that fits additional constraints imposed, thereby degrading the accuracy of the model.
Some researchers have tried to build a model from just one frontal image based on priori information on depth. However, the resulting model quality in this case is far from being photorealistic.
An important issue in face modeling is the automated extraction of facial features. Various facial feature detection methods such as appearance-based modeling of facial features, human vision-based algorithms, search for specific edge and low-intensity pixel patterns, use of morphological operators, etc. have been suggested. Despite the obvious progress in this field, much work is still needed to improve accuracy and robustness for the creation of “human-quality” face analysis algorithms.